With the development of natural language processing techniques(NLP), automatic diagnosis of eye diseases using ophthalmology electronic medical records (OEMR) has become possible. It aims to evaluate the condition of both eyes of a patient respectively, and we formulate it as a particular multi-label classification task in this paper. Although there are a few related studies in other diseases, automatic diagnosis of eye diseases exhibits unique characteristics. First, descriptions of both eyes are mixed up in OEMR documents, with both free text and templated asymptomatic descriptions, resulting in sparsity and clutter of information. Second, OEMR documents contain multiple parts of descriptions and have long document lengths. Third, it is critical to provide explainability to the disease diagnosis model. To overcome those challenges, we present an effective automatic eye disease diagnosis framework, NEEDED. In this framework, a preprocessing module is integrated to improve the density and quality of information. Then, we design a hierarchical transformer structure for learning the contextualized representations of each sentence in the OEMR document. For the diagnosis part, we propose an attention-based predictor that enables traceable diagnosis by obtaining disease-specific information. Experiments on the real dataset and comparison with several baseline models show the advantage and explainability of our framework.

With the evergrowing sizes of pre-trained models (PTMs), it has been an emerging practice to only provide the inference APIs for users, namely model-as-a-service (MaaS) setting. To adapt PTMs with model parameters frozen, most current approaches focus on the input side, seeking for powerful prompts to stimulate models for correct answers. However, we argue that input-side adaptation could be arduous due to the lack of gradient signals and they usually require thousands of API queries, resulting in high computation and time costs. In light of this, we present Decoder Tuning (DecT), which in contrast optimizes task-specific decoder networks on the output side. Specifically, DecT first extracts prompt-stimulated output scores for initial predictions. On top of that, we train an additional decoder network on the output representations to incorporate posterior data knowledge. By gradient-based optimization, DecT can be trained within several seconds and requires only one PTM query per sample. Empirically, we conduct extensive natural language understanding experiments and show that DecT significantly outperforms state-of-the-art algorithms with a $10^3\times$ speed-up.

Metric-based meta-learning is one of the de facto standards in few-shot learning. It composes of representation learning and metrics calculation designs. Previous works construct class representations in different ways, varying from mean output embedding to covariance and distributions. However, using embeddings in space lacks expressivity and cannot capture class information robustly, while statistical complex modeling poses difficulty to metric designs. In this work, we use tensor fields (``areas'') to model classes from the geometrical perspective for few-shot learning. We present a simple and effective method, dubbed hypersphere prototypes (HyperProto), where class information is represented by hyperspheres with dynamic sizes with two sets of learnable parameters: the hypersphere's center and the radius. Extending from points to areas, hyperspheres are much more expressive than embeddings. Moreover, it is more convenient to perform metric-based classification with hypersphere prototypes than statistical modeling, as we only need to calculate the distance from a data point to the surface of the hypersphere. Following this idea, we also develop two variants of prototypes under other measurements. Extensive experiments and analysis on few-shot learning tasks across NLP and CV and comparison with 20+ competitive baselines demonstrate the effectiveness of our approach.

图对比度学习（GCL）一直是图形自学学习的新兴解决方案。 GCL的核心原理是在正视图中降低样品之间的距离，但在负视图中增加样品之间的距离。在实现有希望的性能的同时，当前的GCL方法仍然受到两个局限性：（1）增强的不可控制的有效性，该图扰动可能会产生针对语义和图形数据的特征流程的无效视图； （2）不可靠的二进制对比理由，对于非欧几里得图数据而言，难以确定构造观点的积极性和负面性。为了应对上述局限性，我们提出了一个新的对比度学习范式，即图形软对比度学习（GSCL），该范例通过排名的社区无需任何增强和二进制对比符合性，在较细性的范围内进行对比度学习。 GSCL建立在图接近的基本假设上，即连接的邻居比遥远的节点更相似。具体而言，我们在配对和列表的封闭式排名中，以保留附近的相对排名关系。此外，随着邻里规模的指数增长，考虑了更多的啤酒花，我们提出了提高学习效率的邻里抽样策略。广泛的实验结果表明，我们提出的GSCL可以始终如一地在各种公共数据集上实现与GCL相当复杂的各种公共数据集的最新性能。

资金机构在很大程度上依赖于领域专家与研究建议之间的主题匹配来分配提案审查员。随着建议越来越跨学科，概述提案的跨学科性质是一项挑战，此后，找到具有适当专业知识的专家审阅者。解决这一挑战的重要步骤是准确对建议的跨学科标签进行分类。现有的方法论和申请相关文献，例如文本分类和提案分类，不足以共同解决跨学科建议数据引入的三个关键独特问题：1）提案的纪律标签的层次结构，谷物，例如，从信息科学到AI，再到AI的基础。 2）在提案中起着不同作用的各种主要文本部分的异质语义； 3）提案的数量在非学科和跨学科研究之间存在不平衡。我们可以同时解决该提案的跨学科性质时的三个问题吗？为了回答这个问题，我们提出了一个层次混音多标签分类框架，我们称之为H-Mixup。 H-Mixup利用基于变压器的语义信息提取器和基于GCN的跨学科知识提取器来解决第一期和第二个问题。 H-Mixup开发了Wold级混音，Word级cutmix，歧管混音和文档级混音的融合训练方法，以解决第三期。

任务概括是自然语言处理（NLP）的漫长挑战。最近的研究试图通过将NLP任务映射到人类可读的提示形式中来提高预训练语言模型的任务概括能力。但是，这些方法需要费力且不灵活的提示，并且在同一下游任务上的不同提示可能会获得不稳定的性能。我们提出了统一的架构提示，这是一种灵活且可扩展的提示方法，该方法会根据任务输入架构自动自动自定义每个任务的可学习提示。它在任务之间建模共享知识，同时保持不同任务架构的特征，从而增强任务概括能力。架构提示采用每个任务的明确数据结构，以制定提示，因此涉及几乎没有人类的努力。为了测试模式提示的任务概括能力，我们对各种一般NLP任务进行基于模式提示的多任务预训练。该框架在从8种任务类型（例如QA，NLI等）的16个看不见的下游任务上实现了强劲的零射击和很少的概括性能。此外，全面的分析证明了每个组件在架构提示中的有效性，其在任务组成性方面的灵活性以及在全DATA微调设置下提高性能的能力。

实体链接旨在将模棱两可的提及与知识库中的相应实体联系起来，这对于各种下游应用程序是重要的，例如知识库完成，问题答案和信息提取。尽管已经致力于这项任务，但这些研究中的大多数遵循以下假设，即可以使用大规模标记的数据。但是，当由于劳动密集型注释工作而导致的标记数据不足以针对特定领域时，现有算法的性能将遭受无法忍受的下降。在本文中，我们努力解决了几个弹药实体链接的问题，这只需要最少的标记数据，并且在实际情况下更为实用。具体而言，我们首先提出了一种新颖的弱监督策略，以基于提及的重写生成非平凡的合成实体对。由于合成数据的质量对有效的模型训练有关键的影响，因此我们进一步设计了一种元学习机制，以自动为每个合成实体对分配不同的权重。通过这种方式，我们可以深刻利用丰富而宝贵的语义信息，从而在几个射击设置下得出训练有素的实体链接模型。现实世界数据集上的实验表明，所提出的方法可以广泛改善最新的几杆实体链接模型，并在只有少量标记的数据可用时实现令人印象深刻的性能。此外，我们还展示了模型可传递性的出色能力。

通过微调调整大型预训练模型（PTM）会施加过刺激的计算和存储负担。对参数有效调整（PET）的最新研究发现，与常规微调相比，仅优化以PTM为条件的一小部分参数才能产生PAR性能。通常，PET方法精确设计参数有效的模块（PET模块）可以应用于PTMS内部的任意细粒位置。但是，这些细粒度位置的有效性很大程度上依赖于复杂的手动指定，因此通常会产生次优的结果。与手动指定相反，我们以自动方式探索构建宠物模块。我们将自动\ textbf {s} earch \ textbf {s} parse \ textbf {s} \ textbf {p} arameter- \ textbf {e} fficbf {e} fficient \ textbf {t textbf {t} uning（s $^3 $ pet） 。基于各种PET方法的统一框架，S $^3 $ PET通过双层优化进行了可区分的PET结构搜索，并提出了移动的全局Sigmoid方法，以明确控制可训练的参数的数量。广泛的实验表明，S $^3 $ PET超过了具有较低训练参数的手册和随机结构。搜索结构可保留99 \％的微调性能，具有0.01 \％可训练的参数。此外，S $^3 $ PET的优势通过极低的训练参数预算（0.0009 \％$ \ sim $ 0.01 \％）进行扩增。搜索结构是可转移和解释的，为PET方法的未来设计提供了建议和指导。

Question Answering (QA) is a longstanding challenge in natural language processing. Existing QA works mostly focus on specific question types, knowledge domains, or reasoning skills. The specialty in QA research hinders systems from modeling commonalities between tasks and generalization for wider applications. To address this issue, we present ProQA, a unified QA paradigm that solves various tasks through a single model. ProQA takes a unified structural prompt as the bridge and improves the QA-centric ability by structural prompt-based pre-training. Through a structurally designed prompt-based input schema, ProQA concurrently models the knowledge generalization for all QA tasks while keeping the knowledge customization for every specific QA task. Furthermore, ProQA is pre-trained with structural prompt-formatted large-scale synthesized corpus, which empowers the model with the commonly-required QA ability. Experimental results on 11 QA benchmarks demonstrate that ProQA consistently boosts performance on both full data fine-tuning, few-shot learning, and zero-shot testing scenarios. Furthermore, ProQA exhibits strong ability in both continual learning and transfer learning by taking the advantages of the structural prompt.

快速学习已成为现代自然语言处理的新范式，它直接适应培训的语言模型（PLMS）到$ CLOZE $ -Style预测，自回归建模或序列到序列生成，从而导致各种任务的表现。但是，尚未提出及时学习的标准实施框架，以及大多数现有的及时学习码条，通常是不受管制的，仅为特定方案提供有限的实现。由于有许多细节，例如模板策略，初始化策略和语言化策略等，因此需要在快速学习中考虑，从业者面临障碍，以便快速调整所需的迅速学习方法到他们的应用程序。在本文中，我们展示了{OpenPrompt}，一个统一的易于使用的工具包，可以通过PLMS快速学习。 OpenPrompt是一项研究型框架，配备了效率，模块化和可扩展性，其组合性允许自由地将不同的PLMS，任务格式和提示模块组合在统一的范例中。用户可以宽松地部署快速学习框架，并在没有约束的情况下在不同的NLP任务上评估它们的泛化。 OpenPrompt在{\ url {https://github.com/thunlp/openprompt}}上公开发布。